Markov Chain Monte Carlo Estimation Research Articles

Efficiency comparison of MCMC and Transport Map Bayesian posterior estimation for structural health monitoring

In this paper, an alternative to solving Bayesian inverse problems for structural health monitoring based on a variational formulation with so-called transport maps is examined. The Bayesian inverse formulation is a widely used tool in structural health monitoring applications. While Markov Chain Monte Carlo (MCMC) methods are often implemented in these settings, they come with the problem of using many model evaluations, which in turn can become quite costly. We focus here on recent developments in the field of transport theory, where the problem is formulated as finding a deterministic, invertible mapping between some easy to evaluate reference density and the posterior. The resulting variational formulation can be solved with integration and optimization methods. We develop a general formulation for the application of transport maps to vibration-based structural health monitoring. Further, we study influences of different integration approaches on the efficiency and accuracy of the transport map approach and compare it to the Transitional MCMC algorithm, a widely used method for structural identification. Both methods are applied to a lower-dimensional dynamic model with uni- and multi-modal properties, as well as to a higher-dimensional neural network surrogate system of an airplane structure. We find that transport maps have a significant increase in accuracy and efficiency, when used in the right circumstances.

Can the Phillips curve provide answers to current high inflation rates

Motivated by the weak response of the inflation rate to the tight labor market, the Federal Reserve (Fed) Bank stepped away from its longstanding preemptive policy to fight inflation as the economy approached full employment. This paper aims to reevaluate the relationships of four measures of inflation with the unemployment rate. We accomplish this task by allowing the Phillips curve (PC) slope to vary over time and depend on the magnitude of the unemployment gap. Assessed by both PC convex specification assessment and the Markov chain Monte Carlo (MCMC) estimation method, we confirm the convexity of the PC when the unemployment gap is negative. In addition, we show that the slope of the PC more than doubled when we allow the coefficient on the economic slack to vary over time. Thus, our study shows that PC is still a relevant tool for guiding monetary policy.

Gaussian Process Vector Autoregressions and Macroeconomic Uncertainty

We develop a nonparametric multivariate time series model that remains agnostic on the precise relationship between a (possibly) large set of macroeconomic time series and their lagged values. The main building block of our model is a Gaussian process prior on the functional relationship that determines the conditional mean of the model, hence, the name of Gaussian process vector autoregression (GP-VAR). A flexible stochastic volatility specification is used to provide additional flexibility and control for heteroscedasticity. Markov chain Monte Carlo (MCMC) estimation is carried out through an efficient and scalable algorithm which can handle large models. The GP-VAR is used to analyze the effects of macroeconomic uncertainty, with a particular emphasis on time variation and asymmetries in the transmission mechanisms.

Two-Stage Limited-Information Estimation for Structural Equation Models of Round-Robin Variables

We propose and demonstrate a new two-stage maximum likelihood estimator for parameters of a social relations structural equation model (SR-SEM) using estimated summary statistics (Σ^) as data, as well as uncertainty about Σ^ to obtain robust inferential statistics. The SR-SEM is a generalization of a traditional SEM for round-robin data, which have a dyadic network structure (i.e., each group member responds to or interacts with each other member). Our two-stage estimator is developed using similar logic as previous two-stage estimators for SEM, developed for application to multilevel data and multiple imputations of missing data. We demonstrate out estimator on a publicly available data set from a 2018 publication about social mimicry. We employ Markov chain Monte Carlo estimation of Σ^ in Stage 1, implemented using the R package rstan. In Stage 2, the posterior mean estimates of Σ^ are used as input data to estimate SEM parameters with the R package lavaan. The posterior covariance matrix of estimated Σ^ is also calculated so that lavaan can use it to calculate robust standard errors and test statistics. Results are compared to full-information maximum likelihood (FIML) estimation of SR-SEM parameters using the R package srm. We discuss how differences between estimators highlight the need for future research to establish best practices under realistic conditions (e.g., how to specify empirical Bayes priors in Stage 1), as well as extensions that would make 2-stage estimation particularly advantageous over single-stage FIML.

Estimating and Calibrating Markov Chain Sample Error Variance

Markov chain Monte Carlo (MCMC) methods are a powerful and versatile tool with applications spanning a wide spectrum of fields, including Bayesian inference, computational biology, and physics. One of the key challenges in applying MCMC algorithms is to deal with estimation error. The main result in this article is a closed form, non-asymptotic solution for the sample error variance of a single MCMC  estimate. Importantly, this result assumes that the state-space is finite and discrete. We demonstrate with examples how this result can help estimate and calibrate MCMC estimation error variance in the more general case, when the state-space is continuous and/or unbounded.

Comparison of distance-based spatial weight matrix in modeling Internet signal strengths in Tasikmalaya regency using logistic spatial autoregressive model

To ensure that national development objectives in rural areas are achieved evenly and sustainably, the Government of Indonesia applies the principles of Village Sustainable Development Goals (SDGs), which are derivative programs of SDGs. One of the indicators in measuring the progress and independence of villages in Indonesia is the availability of cellular phone signal access. Cellular phone signals have a vital role because most internet users in Indonesia rely on mobile data connections from cellular operators. However, the signal emitted by a provider tower has a limited range. According to the data of the Developing Villages Index in 2022, Tasikmalaya Regency is one of the regencies with the highest number of villages that have weak signal strength in West Java Province, Indonesia. To examine the effect of distance and height difference between the placement of the nearest provider tower and the location of the Village Office on the internet signal strength category in Tasikmalaya Regency, Logistic Spatial Autoregressive modeling is needed. In this study, the Bayesian Markov-Chain Monte Carlo estimation method was used, because it has advantages in flexibility and computational efficiency. In spatial modeling, there is a spatial weight matrix determined by the researcher’s understanding of the observed phenomenon. The variable observed in this study is signal strength, which has an orientation at a distance. However, there are several types of distance-based spatial weight matrices, such as K-nearest neighbor, radial distance, power distance, and exponential distance. To determine the most suitable distance-based spatial weight matrix in internet signal strength modeling, the four (4) weight matrices were compared based on the goodness of fit measure models, calculated from the confusion matrix. The results of the analysis showed that the radial distance weight matrix with a threshold distance of d = 1.7km is the most suitable use of distance-based spatial weight matrix in internet signal modeling in Tasikmalaya Regency. The weight matrix exerted a positive spatial autocorrelation effect of 57.141%. In addition, the height difference factor between the location of the provider tower with the location of the village office has a greater effect than the horizontal distance.

Nonlinearities in macroeconomic tail risk through the lens of big data quantile regressions

SummaryModeling and predicting extreme movements in GDP is notoriously difficult, and the selection of appropriate covariates and/or possible forms of nonlinearities are key in obtaining precise forecasts. In this paper, our focus is on using large datasets in quantile regression models to forecast the conditional distribution of US GDP growth. To capture possible nonlinearities, we include several nonlinear specifications. The resulting models will be huge dimensional, and we thus rely on a set of shrinkage priors. Since Markov chain Monte Carlo estimation becomes slow in these dimensions, we rely on fast variational Bayes approximations to the posterior distribution of the coefficients and the latent states. We find that our proposed set of models produces precise forecasts. These gains are especially pronounced in the tails. Using Gaussian processes to approximate the nonlinear component of the model further improves the good performance, in particular in the right tail.

Evo‐Scope: Fully automated assessment of correlated evolution on phylogenetic trees

Abstract Correlated evolution describes how multiple biological traits evolve together. Recently developed methods provide increasingly detailed results of correlated evolution, sometimes at elevated computational costs. Here, we present evo‐scope, a fast and fully automated pipeline with minimal input requirements to compute correlation between discrete traits evolving on a phylogenetic tree. Notably, we improve two of our previously developed tools that efficiently compute statistics of correlated evolution to characterize the nature, such as synergy or antagonism, and the strength of the interdependence between the traits. Furthermore, we improved the running time and implemented several additional features, such as genetic mapping, Bayesian Markov Chain Monte Carlo estimation, consideration of missing data and phylogenetic uncertainty. As an application, we scan a publicly available penicillin resistance data set of Streptococcus pneumoniae and characterize genetic mutations that correlate with antibiotic resistance. The pipeline is accessible both as a self‐contained Github repository (https://github.com/Maxime5G/EvoScope) and through a graphical galaxy interface (https://galaxy.pasteur.fr/u/maximeg/w/evoscope).

Hedonic real estate price estimation with the spatiotemporal geostatistical model

This study argues that the spatiotemporal geostatistical model for real estate prices, which accounts for and incorporates spatial autocorrelation, can be estimated successfully using the Bayesian Markov Chain Monte Carlo (MCMC) estimation. While this procedure often encounters difficulty in calculating probabilistic densities in the Metropolis–Hastings (MH) algorithm, this study introduces a feasible and practical estimation method, providing useful estimated parameters for the model. Using single-family house transaction data, we show that ordinary estimations of real estate prices, with respect to certain explanatory variables, may lead to the underestimation of standard errors of coefficients for explanatory variables with spatial effects unless spatial autocorrelation is controlled for. Our model also makes it possible to obtain accurate in-sample predictions and moderately improved out-of-sample predictions for real estate prices. This study further estimates a “decay rate:” a diminishing correlation between real estate prices and increasing distance, showing that geographical proximities are likely to have an important impact on real estate prices, especially at a range under 600 m.

Constraints on cosmological model in Weyl-Cartan spacetime from astronomical measurements

In order to give some lights on the issue of Hubble tension, alternative to Riemannian spacetime, we investigate a cosmological model in Weyl-Cartan spacetime by astronomical measurements, where a new density parameter Ω ϕ is introduced to denote the non-Riemannian spacetime structure, which scales as ∼(1 + z)6. Based on Markov Chain Monte Carlo estimation and taking astronomical measurements of cosmic microwave background (Planck 2018), baryon acoustic oscillations and Type Ia supernova (Pantheon), model parameters of our considering case are constrained. Our results show that Hubble tension cannot be alleviated in Weyl-Cartan cosmology. However, constraints on the parameter Ω ϕ can not be distinguished from ΛCDM model in 1σ confidence region. Moreover, the space of the Universe is infinite and unbounded, and an upper limit of the model parameter Ω ϕ is also obtained. Additionally, it is worth noting that results obtained in this work are valid for models that the scaling behavior of density parameters is same to our considering case.

A SAS Macro for Automated Stopping of Markov Chain Monte Carlo Estimation in Bayesian Modeling with PROC MCMC

A crucial challenge in Bayesian modeling using Markov chain Monte Carlo (MCMC) estimation is to diagnose the convergence of the chains so that the draws can be expected to closely approximate the posterior distribution on which inference is based. A close approximation guarantees that the MCMC error exhibits only a negligible impact on model estimates and inferences. However, determining whether convergence has been achieved can often be challenging and cumbersome when relying solely on inspecting the trace plots of the chain(s) or manually checking the stopping criteria. In this article, we present a SAS macro called %automcmc that is based on PROC MCMC and that automatically continues to add draws until a user-specified stopping criterion (i.e., a certain potential scale reduction and/or a certain effective sample size) is reached for the chain(s).

Integrative Bayesian tensor regression for imaging genetics applications.

Identifying biomarkers for Alzheimer's disease with a goal of early detection is a fundamental problem in clinical research. Both medical imaging and genetics have contributed informative biomarkers in literature. To further improve the performance, recently, there is an increasing interest in developing analytic approaches that combine data across modalities such as imaging and genetics. However, there are limited methods in literature that are able to systematically combine high-dimensional voxel-level imaging and genetic data for accurate prediction of clinical outcomes of interest. Existing prediction models that integrate imaging and genetic features often use region level imaging summaries, and they typically do not consider the spatial configurations of the voxels in the image or incorporate the dependence between genes that may compromise prediction ability. We propose a novel integrative Bayesian scalar-on-image regression model for predicting cognitive outcomes based on high-dimensional spatially distributed voxel-level imaging data, along with correlated transcriptomic features. We account for the spatial dependencies in the imaging voxels via a tensor approach that also enables massive dimension reduction to address the curse of dimensionality, and models the dependencies between the transcriptomic features via a Graph-Laplacian prior. We implement this approach via an efficient Markov chain Monte Carlo (MCMC) computation strategy. We apply the proposed method to the analysis of longitudinal ADNI data for predicting cognitive scores at different visits by integrating voxel-level cortical thickness measurements derived from T1w-MRI scans and transcriptomics data. We illustrate that the proposed imaging transcriptomics approach has significant improvements in prediction compared to prediction using a subset of features from only one modality (imaging or genetics), as well as when using imaging and transcriptomics features but ignoring the inherent dependencies between the features. Our analysis is one of the first to conclusively demonstrate the advantages of prediction based on combining voxel-level cortical thickness measurements along with transcriptomics features, while accounting for inherent structural information.

Generalized Method of Moments Estimation of Realized Stochastic Volatility Model

The purpose of this paper is to study the generalized method of moments (GMM) estimation procedures of the realized stochastic volatility model; we give the moment conditions for this model and then obtain the estimation of parameters. Then, we apply these moment conditions to the realized stochastic volatility model to improve the volatility prediction effect. This paper selects the Shanghai Composite Index (SSE) as the original data of model research and completes the volatility prediction under a realized stochastic volatility model. Markov chain Monte Carlo (MCMC) estimation and quasi-maximum likelihood (QML) estimation are applied to the parameter estimation of the realized stochastic volatility model to compare with the GMM method. And the volatility prediction accuracy of these three different methods is compared. The results of empirical research show that the effect of model prediction using the parameters obtained by the GMM method is close to that of the MCMC method, and the effect is obviously better than that of the quasi-maximum likelihood estimation method.

ESPRESSO: topological clustering of single-cell transcriptomics data to reveal informative genes for spatio–temporal architectures of cells

BackgroundBioinformatics capability to analyze spatio–temporal dynamics of gene expression is essential in understanding animal development. Animal cells are spatially organized as functional tissues where cellular gene expression data contain information that governs morphogenesis during the developmental process. Although several computational tissue reconstruction methods using transcriptomics data have been proposed, those methods have been ineffective in arranging cells in their correct positions in tissues or organs unless spatial information is explicitly provided.ResultsThis study demonstrates stochastic self-organizing map clustering with Markov chain Monte Carlo calculations for optimizing informative genes effectively reconstruct any spatio–temporal topology of cells from their transcriptome profiles with only a coarse topological guideline. The method, eSPRESSO (enhanced SPatial REconstruction by Stochastic Self-Organizing Map), provides a powerful in silico spatio–temporal tissue reconstruction capability, as confirmed by using human embryonic heart and mouse embryo, brain, embryonic heart, and liver lobule with generally high reproducibility (average max. accuracy = 92.0%), while revealing topologically informative genes, or spatial discriminator genes. Furthermore, eSPRESSO was used for temporal analysis of human pancreatic organoids to infer rational developmental trajectories with several candidate ‘temporal’ discriminator genes responsible for various cell type differentiations.ConclusionseSPRESSO provides a novel strategy for analyzing mechanisms underlying the spatio–temporal formation of cellular organizations.

A sequential exploratory diagnostic model using a Pólya-gamma data augmentation strategy.

Cognitive diagnostic models provide a framework for classifying individuals into latent proficiency classes, also known as attribute profiles. Recent research has examined the implementation of a Pólya-gamma data augmentation strategy binary response model using logistic item response functions within a Bayesian Gibbs sampling procedure. In this paper, we propose a sequential exploratory diagnostic model for ordinal response data using a logit-link parameterization at the category level and extend the Pólya-gamma data augmentation strategy to ordinal response processes. A Gibbs sampling procedure is presented for efficient Markov chain Monte Carlo (MCMC) estimation methods. We provide results from a Monte Carlo study for model performance and present an application of the model.

Longitudinal joint modeling for assessing parallel interactive development of latent ability and processing speed using responses and response times.

To measure the parallel interactive development of latent ability and processing speed using longitudinal item response accuracy (RA) and longitudinal response time (RT) data, we proposed three longitudinal joint modeling approaches from the structural equation modeling perspective, namely unstructured-covariance-matrix-based longitudinal joint modeling, latent growth curve-based longitudinal joint modeling, and autoregressive cross-lagged longitudinal joint modeling. The proposed modeling approaches can not only provide the developmental trajectories of latent ability and processing speed individually, but also exploit the relationship between the change in latent ability and processing speed through the across-time relationships of these two constructs. The results of two empirical studies indicate that (1) all three models are practically applicable and have highly consistent conclusions in terms of the changes in ability and speed in the analysis of the same data set, and (2) additional analysis of the RT data and acquisition of individual processing speed measurements can reveal the parallel interactive development phenomena that are difficult to detect using RA data alone. Furthermore, the results of our simulation study demonstrate that the proposed Bayesian Markov chain Monte Carlo estimation algorithm can ensure accurate model parameter recovery for all three proposed longitudinal joint models. Finally, the implications of our findings are discussed from the research and practice perspectives.

A Bayesian Nonparametric Latent Space Approach to Modeling Evolving Communities in Dynamic Networks

The evolution of communities in dynamic (time-varying) network data is a prominent topic of interest. A popular approach to understanding these dynamic networks is to embed the dyadic relations into a latent metric space. While methods for clustering with this approach exist for dynamic networks, they all assume a static community structure. This paper presents a Bayesian nonparametric model for dynamic networks that can model networks with evolving community structures. Our model extends existing latent space approaches by explicitly modeling the additions, deletions, splits, and mergers of groups with a hierarchical Dirichlet process hidden Markov model. Our proposed approach, the hierarchical Dirichlet process latent position cluster model (HDP-LPCM), incorporates transitivity, models both individual and group level aspects of the data, and avoids the computationally expensive selection of the number of groups required by most popular methods. We provide a Markov chain Monte Carlo estimation algorithm and demonstrate its ability to detect evolving community structure in a network of military alliances during the Cold War and a narrative network constructed from the Game of Thrones television series.

An empirical analysis of participation in international environmental agreements

This study investigates the determinants of participation in environmental agreements, with a special focus on lobbying and regional agreements. To this end, we collated the largest ratification dataset in the literature and identified all countries eligible for membership in each agreement. This feature allows to correct the identification bias associated with the misidentification of the risk set when regional agreements are included in the regression sample. We also improve the treatment of unobserved heterogeneity by using a multilevel survival approach and Markov Chain Monte Carlo (MCMC) estimator. Overall, we find that countries’ participation choices are interrelated and primarily driven by the agreements’ characteristics. Our findings indicate that regional agreements are two and half times more likely to be ratified than global agreements and that environmental lobbying positively affects participation in environmental agreements, while the effect of industrial lobbying is statistically insignificant. This result is robust to changes in specification and proxies used. Our results motivate several policy suggestions. We emphasise regional agreements’ capacity to deliver higher participation than global agreements and the importance of securing the early ratification of key players.

Dependence Modelling of Lifetimes in Egyptian Families

In this study, we analyse a large sample of Egyptian social pension data which covers, by law, the policyholder’s spouse, children, parents and siblings. This data set uniquely enables the study and comparison of pairwise dependence between multiple familial relationships beyond the well-known husband and wife case. Applying Bayesian Markov Chain Monte Carlo (MCMC) estimation techniques with the two-step inference functions for margins (IFM) method, we model dependence between lifetimes in spousal, parent–child and child–parent relationships, using copulas to capture the strength of association. Dependence is observed to be strongest in child–parent relationships and, in comparison to the high-income countries of data sets previously studied, of lesser significance in the husband and wife case, often referred to as broken-heart syndrome. Given the traditional use of UK mortality tables in the modelling of mortality in Egypt, the findings of this paper will help to inform appropriate mortality assumptions specific to the unique structure of the Egyptian scheme.

Lindley procedure and MCMC technique in Bayesian estimation for Kumaraswamy Weibull distribution

In this study, a comparison between three methods for estimating unknown parameters of the Kumaraswamy Weibull distribution for different sample sizes of type II censoring data is presented. Specifically, we compare the behaviors of maximum likelihood estimates, Lindley and Markov chain Monte Carlo (MCMC) estimates as Bayesian estimates. We have not found any work on this topic after reviews of the literature except one with little information about the inference of this important distribution. The simplest form for Lindley approximation of the posterior mean is proposed and approximate closed forms of acceptable Bayes estimates for the models of multi-parameters such as Kumaraswamy Weibull distribution is derived. A Monte Carlo simulation is conducted to investigate the performances of the proposed estimators. Finally, three real data examples are analyzed to illustrate the application possibility of the different proposed estimation methods. The results reveal that, although, good performance of the approximate forms of Lindley estimators, the estimators resulting in the MCMC technique are better in the sense of the mean squared errors.